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ABSTRACT 

The Even Start program was authorized by Congress in 
1988 to provide early childhood, adult, and parenting education to 
participating families in a family literacy initiative. This document 
provides an executive summary of the National Evaluation of Even 
Start, which assessed services, participants and effectiveness. The 
summary highlights topics such as the program's organization, 
implementation, and effects. Among the findings summarized are the 
following: (1) in its first 4 years, the program served more than 
20,000 families, most of whom had incomes under $10,000 per year and 
were headed by parents who did not complete high school; (2) the cost 
was about $2,500 per family per year; and (3) the program produced 
effects such as helping parents attain GEDs and improving their 
literacy skills, and helping children increase their school readiness 
and vocabulary. The summary concludes by noting lessons learned from 
the program, specifically the importance of a high level of program 
participation for parents and children and the importance of parent 
interest as a way to improve children's gains. (JW) 
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National Evaluation of the Even Start Family 
Literacy Program: Final Report 

Executive Summary 



Highlights 

Authorized by Congress in 1 988, the Even Start Family Literacy Program is designed to 
improve the educational opportunities of the nation's children and adults by providing 
three core services to participating families: early childhood education, adult education, 
and parenting education. Federal Even Start funds are used for coordinating existing 
services as well as for the direct provision of services which are not locally available. 



Implementation of Even Start 



During its first four years Even Start provided for the implementation 
of several hundred projects which delivered family literacy services 
to more than 20,000 families at a Federal cost of about $2,500 per 
family per year, (see Chapter 4, p.41 and Chapter 12, p. 231 -2361 

Even Start served its intended population. Of all Even Start adults 
served, 79 percent did not complete high school, and 66 percent of 
Even Start families had total annual income under $10,000. The 
average adult entered Even Start with the literacy skills of a high 
school student, while the average three- or four-year-old child who 
entered Even Start scored at the 9th percentile on a nationally- 
normed vocabulary test, [see Chapter 4, p.39-70] 

The average Even Start family participated in the program for seven 
months and received an average of 13.5 hours of adult education, 
6.5 hours of parenting education, and 26 hours of early childhood 
education each month, [see Chapter 6, p. 137-1 45] 



Virtually all Even Start projects offered the three required core 
instructional services of adult education, parenting education, and 
early childhood education. They also offered some instructional 
services in a home-based setting, some services to parents and 
children together, and appropriate support services to enable families 
to participate fully in Even Start's core services, [see Chapter 5, 
p. 72-81] 
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Effect of Even Start on Program Participation 



Adults and children in Even Start families participated much more 
frequently in educational services than they would have had they 
not been in the program: 91 percent of Even Start families 
participated in adult education vs. an estimated 30-40 percent 
without the program, 95 percent participated in parenting education 
vs. an estimated 8 percent without Even Start, and 98 percent 
participated in early childhood education vs. an estimated 60 
percent in the absence of Even Start, [see Chapter 6, p.127-1311 



Effect of Even Start on Parents 



• Even Start helped many adults attain a GED. Across all 120 
projects in the evaluation, 8 percent of adults who entered Even 
Start without a GED or diploma achieved one. In a randomized 
experimental study of a subset of 5 Even Start projects, significantly 
more adults in Even Start than in the control group attained a GED 
(22 percent vs. 6 percent), [see Chapter 9, p. 192-1 98] 

• Adults who participated in Even Start achieved significant positive 
gains (about 4-5 points, or 1/3 of a standard deviation) on the 
CASAS reading survey (a measure of adult functional literacy in 
reading), gains which are comparable or greater in size than those 
observed in other studies of adult education programs. However, 
families in a control group achieved similar gains, [see Chapter 9, 
p.183-192] 

• Even Start did not produce any measurable effects on parenting 
skills as assessed by self-report measures of the home learning 
environment, parenting activities, parental expectations, sense of 
mastery, and depression; and in-home observations of parents 
reading to their children. This could be due to a weakness in our 
ability to measure these areas, to generally positive parent responses 
and high self-ratings at pretest, or to a lack of program 
effectiveness, [see Chapter 10, p. 202-21 8] 

« Even Start did not produce any measurable effect on family 
resources, income, or employment. It is unlikely that effects in 
these areas would be evident in the relatively short time period 
covered by this study, [see Chapter 11, p. 21 9-230] 



Effect of Even Start on Children 



• Even Start children made learning gains, both in terms of school 
readiness skills and on a vocabulary test. 
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By participating in Even Start, children learned school readiness skills 
such as colors, shapes, and sizes (as measured by the PreSchool 
Inventory) significantly faster than would be expected on the basis 
of normal development. Further, after one year of participation, 
Even Start children had significantly higher scores on the PSI than 
children in the control group. However, control group children 
caught up once they entered preschool or kindergarten, [see Chapter 
8, p. 157-165] 

When they entered the program Even Start children scored quite 
low, at the 9th percentile nationally, on the Peabody Picture 
Vocabulary Test (a test of receptive language). During their 
participation in Even Start, children gained a significant amount on 
the PPVT (so that they scored at the 19th percentile nationally). 
However, children in a control group gained a similar amount, [see 
Chapter 8, p. 165-1 73] 



Relationship Between Amount of Service and Outcomes 



A high level of exposure to program services is important. Adults 
and children with high levels of participation in Even Start's core 
services gained significantly more on tests than adults and children 
with low levels of participation, [see Chapter 8, p.180-181 and 
Chapter 9, p. 189-1 92] 



The extent to which parents took part in parenting education is 
significantly related to gains in children's vocabulary (as measured 
by the PPVT), over and above gains in vocabulary that result from 
children participating in early childhood programs. Families that 
participated in a relatively low-intensity or moderate-intensity 
program had children who scored at the 17th to 19th percentile, 
while families that participated in a relatively high-intensity program 
had children who scored at the 26th percentile [see Chapter 8, p. 
175-180]. 
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To sum up, Even Start has had some positive short-term effects on children and adults, 
although those effects vary greatly across projects. In fact, there is great project-to- 
project variation in all facets of Even Start-program planning and implementation, 
characteristics of families served, participation, and outcomes. Even Start's services did 
result in gains for children and their parents, but on average, the gains are not greater 
than those that similarly motivated families would obtain for themselves using locally 
available services. In order for Even Start to have a larger effect, it must provide services 
more intensively. 

Even Start families that were intensively engaged in core services did better than families 
with lower levels of participation. Further, there are indications that providing parenting 
education to parents has positive effects on their childrens' vocabularies. This type of 
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parent-to-child effect is encouraging in that it is exactly what Even Start hopes to 
produce. While other early childhood programs have demonstrated positive short-term 
effects on children, and some adult education program? have demonstrated positive 
effects on adults, Even Start is one of the few programs that has focused on parent 
literacy, parenting skills, and child development all at the same time. 

In the long run, it is hoped that Even Start's effects on children will be enhanced because 
of the positive changes made in their parents. Follow-up studies of Even Start 
participants could be conducted to determine whether this long-term portion of Even 
Start's theory holds-that positive effects on parents will lead to long-term positive effects 
for children's cognitive development, social-emotional development, and behaviors. 

The remainder of this executive summary expands upon these highlights. A full 
evaluation report is available from the U.S. Department of Education, Planning and 
Evaluation Service, or from Abt Associates Inc. of Cambridge, Massachusetts. 



Background 

The nation's literacy problems have traditionally been addressed through a dual system 
of public and private sector efforts: (1) remediation programs for adults in the form of 
adult education or workplace literacy programs, and (2) prevention programs for children 
through early childhood education efforts such as Head Start. The seeds of a new 
approach were sown in the late 1970s and early 1980s when many of the first "family 
literacy" programs were planned and implemented. Drawing on the experiences of 
existing early intervention and adult literacy programs, family literacy programs are based 
on the beliefs that children's early learning is greatly influenced by their parents, that 
parents must develop and value their own literacy skills in order to support their children's 
educational success, and that parents are their children's first and best teachers. 

In the middle and late 1980s this approach emerged in full force as family Iit3racy 
programs proliferated under a wide range of sponsors including state governments (e.g., 
Kentucky's PACE program), school districts (e.g., the Marin, California, Library Family 
Literacy Program), private foundations (e.g., the Kenan Charitable Trust's Family Literacy 
Program), private corporations (e.g., Stride Rite's Intergenerational Day Care Program), 
and universities (e.g., El Paso State College's Family Literacy Intergenerational English 
Literacy Program). The movement attained national status in 1989 when the federal 
government instituted its family literacy centerpiece, the Even Start program. 



The Even Start Family Literacy Program 

The Even Start Family Literacy Program was authorized by the Elementary and Secondary 
Education Act of 1965 as amended by the Hawkins-Stafford Elementary and Secondary 
School Improvement Amendments of 1988, Part B of Chapter 1 of Title I (P.L. 100-297). 
In 1991, Congress passed the National Literacy Act (P.L. 102-73) which amended the 
Even Start program. According to the law, the Even Start program is intended to: 



Even Start 



4 



( 



Executive Summary 



...improve the educational opportunities of the Nation's children and adults 
by integrating early childhood education and adult education for parents into 
a unified program.... The program shall be implemented through cooperative 
projects that build on existing community resources to create a new range 
of services. (P.L. 100-297, Sec. 1051). 

To participate in Even Start, a family must include an adult who is eligible for adult 
education programs under the Adult Education Act and who is a parent of a child less 
than eight years of age who lives in a Chapter 1 elementary school attendance area. Even 
Start projects must provide participating families with an integrated program of early 
childhood education, adult literacy or basic skills training, and parenting education. The 
program's design is based on the notion that these components build on each other and 
that families need to receive all three services, not just one or two, in order to effect 
lasting change and improve children's school success. 

The U.S. Department of Education began administering Even Start in fiscal year 1989 as 
a demonstration program which provided school districts with discretionary grants for 
family literacy projects, generally for four years. In 1992, the program became 
administered by the states which hold grant competitions and make subgrant awards of 
a minimum of $75,000 per project annually. Exhibit 1 summarizes Even Start's funding 
history. 





Exhibit 1 






Even Start Funding History 




Fiscal Year 


|| Federal Funding 


Number of Projects 


1989 


$14,820,000 


76 


1990 


$24,201,000 


123 


1991 


$49,770,000 


234 


1992 


$70,000,000 


340 


1993 


$89,123,000 


440 


1994 


$91,373,000 


474 


Exhibit reads: In fiscal year 1991, $49.8 million of federal funding was used to provide Even Start 
grants to 234 local projects. 

Source: U.S. Department of Education, Office of Policy and Planning. 
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Evaluation Design 



The Even Start legislation required an independent national evaluation. In January 1990, 
the Office of Policy and Planning in the U.S. Department of Education (ED) awarded a 
contract for that evaluation to Abt Associates Inc., with a subcontract to RMC Research 
Corporation. 

The effectiveness of Even Start was assessed by analyzing data from an experimental 
study ;the In-Depth Study) conducted in a subset of five Even Start projects with about 
200 families that were randomly assigned to be in Even Start or in a control group, and 
from an annual survey (the National Evaluation Information System, or NEIS) of all Even 
Start projects and participating families. Data on program costs were collected from Even 
Gtart grant awards and from a functional cost analysis conducted in 10 fully implemented 
projects participating in the In-Depth Study. Finally, descriptive data on Even Start 
projects, staff, and implementation, and on the characteristics and participation rates of 
families were collected through the NEIS annual survey of all propcts. 

Outcome data for the In-Depth Study were collected on program and control group 
children and adults prior to entering Even Start (when children were about four years of 
age), and twice again at nine months and at 18 months after entering Even Start. 
Families in the In-Depth Study were assessed at each measurement point, even if the 
program families were no longer participating in Even Start. This longitudinal design 
provides an unbiased, if conservative, estimate of program impacts by allowing 
comparisons of gains for randomly-assigned Even Start and control group families . 

Outcome data for the NEIS survey of all Even Start projects were collected on program 
families at entry to Even Start and at the end of the program year. Thus, the NEIS 
outcome data are restricted to the subset of families who remained in the program 
between pretesting and posttesting. Any family that did not participate long enough to 
be posttested had to be omitted from the analysis. Since we expect length of 
participation to be related to positive program effects, we also expect families measured 
in the NEIS to show larger gains than program families measured in the In-Depth Study . 

Findings from the NEIS can be used to generalize only to the population of families that 
remain in the program for several months (long enough to be posttested), while findings 
from the In-Depth study can be generalized to all Even Start families in the selected 
projects, including those that drop out early. Because no control group families were 
included in NEIS, data from this part of the study can only tell us how much was gained 
by Even Start families-not whether those gains are larger or smaller than what would 
have occurred in the absence of Even Start. 

Finally, Even Start recruits and serves families that are willing to participate in the three 
core service areas. Because adults must be willing to take part in two types of 
interventions (parenting education and adult education), it is possible that the Even Start 
population is more motivated to succeed than the populations served by less demanding 
social programs such as Head Start. 
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Program Design 



The Even Start legislation contains language setting forth the major elements that must 
be the basis of each Even Start project. However, grantees are allowed great flexibility 
in devising projects to meet local needs, and Even Start can be regarded as a family 
literacy laboratory in which many different approaches to delivering a common set of 
services are being tested. 

Even Start projects are designed to use and build on existing educational and social 
services. Federal Even Start funds are used for coordinating existing services, as well as 
for the direct provision of services which are not locally available. Many Even Start 
projects use case managers, parent liaisons, or family advocates as key staff in the 
provision of coordinated services. Case managers conduct needs assessments and have 
ongoing contact with families at centers and through home visits. They are responsible 
for direct provision of some services as well as for ensuring that families take advantage 
of other services. Three core Even Start services are required by the legislation: 

• Early childhood education: services to meet the early education 
needs of children from birth through seven years of age, designed 
to enhance development and prepare children for success in school. 

• Adult education: services that develop the basic educational and 
literacy skills of the adult including adult basic education, adult 
secondary education, English as a second language, or preparation 
to attain a General Education Development (GED) certificate. 

• Parent education: services for parents to enhance parent-child 
relationships and help parents understand and support their child's 
growth and development. 

Projects are free to choose the specific instructional strategies used in each core service 
area, although ED regulations require that each family participate in all three areas. Even 
Start projects are required to provide some core services to parents and children in joint 
sessions and to provide some core instructional services in a horr e-based setting. They 
also can decide to focus educational activities for children on a narrower age span than 
the birth through seven range that is targeted by the legislation. Finally, core services 
may be provided either by staff funded through Even Start or by staff at cooperating 
agencies. For example, early childhood education may be provided by a local Head Start 
project and adult education classes might take place at local community colleges. 

In addition to core services, Even Start projects provide "support" services which are 
designed to enable families to participate in core program activities. Examples of support 
services are transportation, child care, and referrals for employment. The Even Start 
legislation requires that support services be obtained from existing providers whenever 
possible, to ensure that Even Start projects avoid duplication of services. 
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Characteristics of Even Start Participants 



Based on data reported for the 1989-90 through the 1992-93 program years, Even Start 
projects are serving the intended population. All participating Even Start households had 
at least one child between birth and age seven, 79 percent of the adults who participated 
in Even Start core services did not complete high school, and 66 percent of Even Start 
families had total annual income (earned income plus public assistance) under $10,000. 
The Even Start population can be further described as follows: 

• 50 percent of Even Start families describe themselves as couples 
with children, 37 percent are single parent households, and 13 
percent have extended families or other living arrangements. 

• 46 percent of Even Start families report job wages as their primary 
source of financial support, while 49 percent report that government 
assistance is their primary source of support. 

• Most adults in Even Start are ages 22-29 (46 percent) or 30-39 (31 
percent). 

• 40 percent of Even Start adults are white, 26 percent are African 
American, 22 percent are Hispanic, 4 percent are Native American, 
and 8 percent are Asian or Pacific Islander. 

• English is the primary language for 66 percent of Even Start adults; 
Spanish is the primary language for 26 percent. 

• Seven percent of the children served by Even Start were identified 
as having a disability. 

• The average age of Even Start children dropped from 4.3 years in 
1989-90 to 3.7 years in 1992-93, reflecting a federal emphasis on 
serving younger children. 

• Even Start children scored low, at the 9th percentile nationally, on 
3 vocabulary test given when they entered the program. 

• Even Start adults attained high school level scores on a functional 
literacy reading test given when thev entered the program. 

LEP Adults. Even Start adults with limited English proficiency (LEP) can be characterized 
as follows: 86 percent were educated outside of the United States, 60 percent did not 
reach the ninth grade, 78 percent were not employed at the time they joined Even Start, 
83 percent had an annual income of less than $15,000, and 18 percent were single 
parents. 

Adults Who Enter with a Diploma or GED. Twenty-one percent of the adults who 
participated in Even Start entered already having attained a high school diploma or a GED. 
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Adult/Child Services Delivered Through Even Start. More than 90 percent of the Even 
Start projects reported that they delivered core services to parents and children together. 
This is an important part of the Even Start model in that it impresses on parents that they 
are key to their child's education. Examples of services delivered to adults and children 
together include reading and story telling, developing readiness skills, social development 
and play, development of gross motor skills, working with numbers, arts/crafts, and 
health/nutrition. 

Support Services. Support services remove barriers tnat could restrict a family's ability 
to participate in Even Start core educational services. Several types of support services 
were provided by 80 percent or more of the projects including transportation, family 
advocacy assistance, nutrition services, counseling services, and child care. 

Cooperative Arrangements. Even Start projects are required to establish cooperative 
arrangements with other agencies to avoid duplicating services. This strategy allows 
optimal use of limited resources so that projects can concentrate on filling service gaps. 
Collaboration and cooperative arrangements were, indeed, a key focus of Even Start 
projects. During the 1992-93 program year, Even Start projects were involved in more 
than 6,000 cooperative arrangements to provide core services, an average of 20 
cooperative arrangements per project. Forty-two percent of the arrangements were for 
parenting education, 27 percent were for adult basic education, and 31 percent were for 
early childhood education. The most common cooperators were "other departments and 
programs within the public schools," "local, county, state or tribal agencies," and 
"postsecondary institutions." 

Implementation Problems. Even Start projects reported several barriers to implementation. 
The most common barriers were difficulties in the recruitment, retention, attendance, and 
motivation of families, problems of communication and coordination with cooperating 
agencies, lack of transportation for families, the unexpectedly severe social service needs 
of the families and community, lack of quality child care, financial problems, staffing 
problems, and problems with facilities and space. 

Staffing Even Start Projects. Most Even Start projects have a project administrator and 
three to five staff who provide instruction in early childhood education and adult 
education. About half of the project administrators have multiple job responsibilities, 
including coordination or supervision in each of the three core services, recruiting families, 
and project evaluation. 



Approximately 54 percent of the staff providing services to Even 
Start families are paid by collaborating agencies, 32 percent are paid 
through the federal grant, and 13 percent are paid with local 
matching funds. Staff from collaborating agencies are most likely to 
provide direct instruction in early childhood educatiot, and adult 
education, although their classrooms may have a small number of 
Even Start participants. 
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• Even Start project administrators have an average of nine years of 
work experience in uarly childhood education programs, four years 
in adult education programs, and six years with parenting education. 

• Early childhood education teachers have an average of eight years 
of work experience in that field, while adult education teachers have 
an average of five years of work experience in adult education 
programs. 

One-third of Even Start staff have a college degree, 20 percent more have a master's 
degree, and an additional 9 percent have a doctorate. Those with a master's degree or 
beyond tend to be project directors, coordinators, or staff involved in the evaluation. 
More adult education teachers (36 percent) have a master's or doctorate than early 
childhood teachers (20 percent) or parenting education teachers (28 percent) . Nearly one- 
quarter of early childhood or parenting education teachers do not have a college degree 
compared to only 9 percent of adult education teachers. 

Even Start staff reported receiving an average of 40 hours of inservice training per year. 
Sessions most frequently focus on curriculum and instruction in the core service areas. 
Staff attended state, regional, and national conferences on family literacy, early childhood 
education, adult education, vocational education, and bilingual education. 



Participation in Even Start 

The great majority of Even Start families participate in early childhood education, 
parenting education, and adult education services. The proportion of families participating 
in all three core services is much higher in mature projects than in projects in their first 
or second year of operation. 

Number of Participating Families. The number of families participating in Even Start has 
increased over time. This is due both to the addition of new projects and to improved 
efficiency among existing projects. During the 1989-90 school year, when Even Start 
began, 76 projects served about 2,500 families. By the 1992-93 school year, the 
program had grown to 340 projects which served about 20,000 families. 

Length of Participation. Although Even Start projects generally are funded for four years, 
relatively few families take part for that amount of time. Of the families that began Even 
Start in 1989-90, 53 percent participated only in that first year, 24 percent participated 
in both the f st and second program years, 13 percent participated in the first three 
program years, and 10 percent participated in all four years. 

Reasons for Leaving. There are many reasons that families end their participation in Even 
Start; some are indicators of program success, while others signal problem areas. 
Seventy percent of the families that left the program indicated a reason for doing so. 
Based on families that reported a reason for leaving, completion of the planned 
educational program or meeting personal objectives was listed by 24 percent of families 
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that exited Even Start. Moving out of Even Start's catchment area was listed for 24 
percent of families that exited. Thirteen percent of the families left because of a general 
lack of interest in the program and a subsequent refusal to participate. Another 12 
percent reported that a family crisis prevented them from participating. Ten percent left 
because they became ineligible due to a change in the family situation, i.e., there was no 
longer an eligible child or adult in the family. Five percent gave a variety of reasons 
including medical problems, work conflicts, pregnancy, scheduling conflicts, child care 
problems, and a lack of transportation. 

Participation in Core Services. The Department of Education requires that all Even Start 
families participate in the three core service areas during their time in the program. 

• Almost all families had a child that participated in early childhood 
education during each year of the study: 90 percent in 1 989-90, 97 
percent in 1990-91, and 98 percent in 1991-92 and 1992-93. 

• Participation rates for parenting education were a little lower: 88 
percent in 1 989-90, 94 percent in 1 990-91 , 93 percent in 1 991-92, 
and 95 percent in 1992-93. 

• At the beginning of Even Start, participation rates were low for adult 
education; 54 percent in 1989-90. This was due, in part, to 
misunderstandings about the definition of legitimate adult education 
services as well as problems convincing adults to participate on a 
regular basis. The Department of Education and local projects 
worked hard to increase these rates to 80 percent in 1990-91, 90 
percent in 1991-92, and 91 percent in 1992-93. 

• The percentage of families participating in all three core services 
increased steadily over the four years of the study, from 46 percent 
to 75 percent to 84 percent to 86 percent. 

Amount of Core Services Received. The typical Even Start family received core services 
in seven different months during their period of enrollment in Even Start. 

• The total amount of service received by the average fmily is 107 
hours for adult education (median of 41 hours), 58 hours for 
parenting education (median of 29 hours), and 232 hours for early 
childhood education (median of 102 hours). There is great variation 
in these numbers across projects. 

• On a monthly basis, the average Even Start family received 13.5 
hours of adult education (median of 9 hours), 6.5 hours of parenting 
education (median of 4 hours), and 26 hours of early childhood 
education (median of 14 hours). 

There is a clear relationship between amount of early childhood education and age of 
child. The average Even Start child less than one year of age received 15.5 hours per 
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month. One-year-olds received 14.3 hours per month, two-year-olds received 20.0 hours 
per month, three-year-olds received 29.2 hours per month, and four-year-olds received 
35.5 hours per month (this is about 25 percent less than the average amount of early 
childhood education received by Head Start children). 

Relationship Between Home-Based Services, Retention, and Amount of Service. There 
is a strong positive relationship between the amount of home-based service provided by 
a project and retention/participation in Even Start. Program retention, defined as the 
percentage of families participating in Even Start for six or more months, increases from 
about 40 percent in projects that provided no home-based services, to about 70 percent 
in projects that provided two-thirds or more of their services in the home. At the same 
time that retention is improved, hours of service are decreased in projects that emphasize 
home-based instruction. That is, the average hours of instruction per month received by 
participating families decreased as the amount of home-based instruction increased. It 
is easy to have high amounts of instruction in a center-based early childhood program 
where children participate as a group, while a home-based program which includes one- 
on-one instruction rarely will be able to provide more than one or two hours per week. 

Effects of Even Start on Services Received 

Our ability to attribute any observed changes in adults or children to Even Start as 
opposed to other factors is dependent on the extent to which Even Start has been able 
to increase families' participation in the three core service areas over what it would have 
been in the absence of Even Start. Data from this study show that Even Start has 
substantially increased participation rates in the three core service areas. 

• We estimate that without Even Start, 30 to 40 percent of the Even 
Start population would take part in adult education. This is 
substantially less than the 91 percent participation rate achieved for 
Even Start families during the 1992-93 program year. 

• For parenting education, participation increases from 8 percent 
without Even Start to 95 percent for families in Even Start. 

• For early childhood education, participation rates are estimated to be 
about 60 percent in the absence of Even Start and 98 percent for 
families in Even Start. 

These increases in service levels indicate that Even Start projects achieved the important 
step of ensuring that program services actually were received by participating families. 

Effects of Even Start on Children 

Children in Even Start acquired school readiness skills earlier than their counterparts in a 
control group. Further, children of adults with high amounts of exposure to parenting 
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education classes gained more on a vocabulary test than children of adults with less 
exposure to parenting education. 

School Readiness. The PreSchool Inventory (PSI) measures school readiness skills (e.g., 
shapes, colors, sizes) and we expect children's scores on the PSI to improve with age, 
simply as a function of maturation. Based on data collected through the NEIS (and from 
many other studies that have used the PSI), we know that children's P?l scores increase 
by about .40 items per month due to normal maturation. 

• By participating in Even Start, children's PSI scores increased at 
more than double the expected rate, by .91 items per month. 

• Data from the In-Depth Study show that Even Start children gained 
more than control group children during their early participation in 
the program. 

• However, follow-up measurements show that after 18 months 
control group children caught up with their Even Start counterparts, 
both because control group children enrolled in preschool or 
kindergarten, and because some Even Start children no longer 
participated in an Even Start early childhood program. 

• Data from the NEIS show a positive relationship between amount of 
participation in early childhood education and gains on the PSI. 

When they were three or four years of age, Even Start children were more likely than 
control group children to participate in early childhood education programs. Because of 
this, they showed an accelerated rate of learning on the PSI, suggesting that they learned 
basic concepts and precursors of kindergarten skills at an earlier age than they would 
have in the absence of the program. However, as control group children reached four and 
five years of age they, too, enrolled in preschool or reached public school age. At that 
point, they learned many of the same skills that Even Start children had learned at a 
younger age. What is not known is the long-term effect of the early boost in learning that 
was experienced by Even Start children. It is possible that since Even Start children 
learned readiness skills prior to entering the public schools, they may have progressed to 
other skills during kindergarten. 

Receptive Vocabulary. The Peabody Picture Vocabulary Test (PPVT) measures hearing 
(receptive) vocabulary. Unlike the PSI. the PPVT has national norms. When given the 
PPVT as a pretest, Even Start children averaged 80 standard score points. This 
corresponds to the 9th percentile nationally and highlights the low verbal skills of children 
at entry to Even Start. 

When compared with national norms, data from the NEIS indicate that Even Start children 
gained more than would be expected without the program; when posttested, they scored 
at the 19th percentile nationally. However, the In-Depth Study shows that Even Start 
children gained no more than their control group counterparts. As was the case for the 
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PSI, data from the NEIS demonstrate a positive relationship between amount of early 
childhood education service and gains on the PPVT. 



Emergent Literacy Skills. The Children's Emergent Literacy Test (CELT) was developed 
by Abt Associates for this evaluation because no existing measure of children's emergent 
literacy skills was ideally suited to the Even Start evaluation. The CELT includes items 
assessing orientation and directionality of text, recognition or letters and punctuation, and 
the purposes of reading. It was administered only in the In-Depth Study, and while Even 
Start children gained a significant amount, there was no program impact because control 
group children also gained on this measure. 



Correlates of Child Effects. Greater exposure to early childhood education is associated 
with greater gains for children on the PSI and PPVT. Just as important, and more relevant 
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found in the home, e.g., books, magazines, newspapers. This finding is consistent with 
the observation that many Even Start projects make a concerted effort to increase the 
number of reading materials in participants' homes, either through loans or donations. 

Parent-Child Reading Task. The In-Depth Study included a newly-developed measure of 
parent-chiid reading interactions. The parent was asked to read a simple book to her child 
while a trained observer used a pre-coded rating form to record five major aspects of 
parent-child interactions: parent describes book to child, parent questions or responds to 
child, chi'd responds, parent's general affect, and child's general affect. There was no 
effect of Even Start on any of the five areas of observation. 

Parental Expectations. In both the In-Depth Study and the NEIS, parents were asked 
questions about how well they expected their child to do in high school and the likelihood 
that their child will graduate from high school. As is the case for such questions in most 
studies, parents generally had high expectations for their children at the pretest. Even 
Start significantly increased parental expectations as assessed by the NEIS but not the 
In-Depth Study. 



Effects of Even Start on Families 

The In-Depth Study included several measures of the effects of Even Start on participating 
families including their perceived social support, adequacy of financial resources, income 
level and sources, and employment status. The NEIS instruments had questions about 
income and employment. There were no significant program effects on any of these 
measures. This is not surprising - these areas are very difficult to change because they 
are affected by circumstances beyond the control of the program, such as the local 
economy and the availability of jobs. It is likely that longer interventions and other 
positive short term impacts will be required in order to achieve significant gains on these 
measures. 

While there are no effects for families on these quantifiable variables, Even Start 
participants described a number of qualitative changes in their lives and the lives of their 
children. Listening to the personal stories of program participants during focus groups 
conducted in ten projects, it became apparent that parents saw positive changes in their 
attitudes and skills. Parents described themselves as moving toward their goals of an 
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resources (e.g., matching funds, in-kind contributions, and the value of referred services), 
in addition to their federal Even Start funds, in order to deliver appropriate services to 
participating families. 

• In 1991-92, the average of $2,503 in federal funding per family was 
augmented by an average of $1,352 in other resources to arrive at 
total resources of $3,855 per Even Start family. Thus, federal Even 
Start funds comprise 65 percent of the total resources used per 
family and other funds comprise 35 percent. 

• Almost two-thirds (64 percent) of all resources were spent on the 
direct provision of services: 31 percent for early childhood 
education, 15 percent for adult education, 9 percent for parenting 
education, and 9 percent for support services. 

• Funds also were spent for program administration and coordination 
(14 percent), evaluation (10 percent), case management and 
recruiting (4 percent), and for other functions (8 percent). 

Even Start costs vary tremendously across projects. As might be expected, projects that 
serve large numbers of families do so at a lower cost per family. In particular, projects 
that serve 100 or more families do so at a federal cost of $1,659 per family, while 
projects that serve 30 or fewer families spend an average of $6,312 in federal Even Start 
funds per family. Also, projects that delegate responsibility for providing core services 
to cooperating agencies have a lower cost ($1,878 per family) than projects that retain 
primary responsibility for providing core services (cost of $5,775 per family). 



Implementation and Use of the Evaluation 

Implementation of the Evaluation. The national Even Start evaluation was implemented 
as a collaborative effort between staff at the U.S. Department of Education, the staff of 
the evaluation contractors, and the staff of each Ever. Start project. It was hoped that 
a greater than usual level of involvement by Even Start grantees would benefit the 
evaluation through an increased investment and level of understanding on the part of 
program implementers, and through a better appreciation of programmatic issues and 
problems on the part of evaluation staff. 

Responsibilities for the national evaluation were divided such that oversight came from 
the Department of Education; design, analysis, reporting, and technical assistance as well 
as data collection for the In-Depth Study were provided by the evaluation contractors; and 
input to the design and analysis, interpretation of findings, and primary responsibility for 
data collection for the NEIS were provided by local Even Start projects. To enable 
projects to undertake their evaluation responsibilities, they were given additional grant 
funds which were used to hire a local evaluator who helped interact with the national 
evaluation, assisted with data collection, and sometimes designed and conducted their 
own local evaluation activities to supplement the national evaluation. 
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Staff from each Even Start project were involved in an early design meeting where 
feedback was obtained on drafts of the data collection forms, and where a working group 
of Even Start project directors and local evaluators was established. Subsequent drafts 
of data collection forms were shared first with the working group, and then with all 
project directors and local evaluators. Training sessions in administering the data 
collection instruments were held for all Even Start projects; additional feedback on forms 
and on data collection problems was provided through formal meetings, mailings, and 
telephone-based technical assistance. 

This approach was seen as experimental, and at the start of the evaluation it was unclear 
whether local project staff would be willing and able to undertake a high-quality data 
collection effort. After four years of experience, it is possible to draw the following 
conclusions about the implementation of the evaluation. 

• Many Even Start projects were comfortable with their role as data 
collectors and were able to turn in high-quality data according to the 
time schedule set forth for the evaluation. Others needed 
substantial assistance in complying with the requirements of the 
evaluation, and several projects were not able to provide data 
according to the hoped-for timetable. 

• Most Even Start projects availed themselves of the evaluation 
technical assistance provided by the evaluation contractor, and 
would have preferred more. 

• Data collection was done by local staff with a wide range of 
backgrounds. In some cases, local evaluators did the data 
collection. In others, project staff (e.g., case managers, early 
childhood teachers) conducted the data collection. 

• The arrangement to have data collected by local project staff has 
the drawbacks of potential bias, lack of attention to important 
details (e.g., matching family identification numbers over time), and 
work-load conflicts between evaluation and programmatic activities. 
The training and technical assistance provided by the national 
contractor worked to ameliorate these problems. On balance, we 
believe that the reliability of the data collection conducted under this 
model is somewhat lower than what would be attained by trained 
researchers, but that the response rates obtained by local staff are 
probably higher than the response rates that would be obtained by 
outside staff. 

• There was great variability in the interest and ability of Even Start 
grantees to collect data using a microcomputer-based system. This 
system worked well for grantees with access to up-to-date 
equipment and staff who understood the basics of computer 
systems. Other grantees struggled and were frustrated with this 
approach, leading to delays in the submission of data. Still others 



Even Start 19 Executive Summary 

© 09 

ERIC 



were not able to use a computer-based system, and submitted data 
on paper forms. 

Use of the Evaluation. The evaluation was an integral part of Even Start's development. 
The Department of Education's requirement to collect a standard set of data on the 
characteristics of program participants, the operations of each project, and the effects of 
Even Start led to implementation of an information system that ensured that grantees had 
common definitions for key programmatic activities. For example, critical definitions such 
as "Who counts as a program participant?" and "What counts as a core service?" were 
clarified by the need to provide data for the evaluation. 

Descriptive data from the evaluation identified achievements and confirmed that Even 
Start projects served the intended population and that projects were successful u\ 
establishing cooperative relationships with local organizations. 

Data from the first year of the evaluation plus ongoing monitoring conducted by the 
Department of Education discovered a need to revise the program's eligibility requirements 
so that a family was not terminated from participation when a parent acquired a GED or 
a child reached age eight. The legislation was changed so that a family may participate 
in Even Start until all family members become ineligible. 

Additionally, the evaluation discovered that a significant number of Even Start projects 
had difficulty persuading adults to participate fully in adult education programs. This led 
the Department of Education to issue non-regulatory guidance as well as provide technical 
assistance in implementing adult education services for the range of skill levels among 
Even Start families. Participation rates in the adult education component of Even Start 
did rise over time, likely as a result of the Department's efforts and increased maturity on 
the part of projects. 

Finally, data from the NEIS were used by individual local projects in preparing local 
evaluations, and proved to be instrumental in helping some projects gain approval by the 
Department of Education's Program Effectiveness Panel. 



Conclusions 

We conclude this report by returning to a subset of the research questions that guided the 
evaluation and briefly summarizing the answers to each question. 

What are Even Start's "best practices"? What types of projects or program elements 
work best under what conditions? 

While evaluating "best practices" directly was beyond the scope of this project, some 
relevant conclusions can be drawn: 

• Program retention is increased in projects that provide home-Dased 
services. 
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• More time spent receiving services is associated with greater gains 
on literacy tests for adults and children. 

Providing home-based services is a good way to increase retention. There is a strong, 
positive relationship between the amount of home-based services and the length of 
program participation. In all three core services, the proportion of families who stay in 
the program more than six months goes up from 40 percent in projects with minimal 
home-based services to 70 percent in projects where a majority of services are provided 
in the home. 

A greater amount of exposure to Even Start's core services (i.e. larger amounts of time 
spent receiving core services) appears to have a positive impact on literacy outcomes for 
children and adults. In other words, more time spent in early childhood education is 
associated with greater school readiness and vocabulary gains for children, and more time 
spent in adult education is associated with greater functional literacy gains for adults. 
The findings about home-based services and amount of service exposure work against 
each other, since it is difficult to deliver large amounts of instructional service in a home- 
based setting. 

Evaluatinrj "best practices" directly was beyond the scope of this evaluation. Anecdotal 
reports from the In-Depth Study site visits identified many innovative instructional 
strategies that were used with parents and children which could benefit from additional 
research. Examples include: 

• Using computers in adult education classrooms to write newsletters 
sharing personal experiences and program activities and to institute 
a pen pal program with other Even Start projects. 

• Incorporating newspapers into parenting and adult education classes 
through daily discussions of current events, distribution of free 
Spanish-language papers, and written homework about newspaper 
articles. 

• Integrating adult and early child education by having parents write 
stories for their children and make books to read at home. 

• Enhancing early childhood classrooms with signs and symbols to 
foster emergent literacy skills. 

• Hiring social workers to provide parenting education and staff 
training. 

• Using a team approach to staffing to facilitate integration of adult 
and child curricula and instruction in the classroom and during home 
visits. 
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How does the program compare to alternative programs addressing the same problem? 
Is it more effective? How do the costs compare? 

This question was not addressed in the current evaluation. However, how Even Start 
compares to alternative proyiams in cost and effectiveness is a question that has been 
asked by researchers and policymakers over the course of the evaluation, and merits 
further study. 

What is the program's impact on its target population and service delivery structure? 

Even Start is not large enough to make an impact on its entire target population. 
However, it has had a broad impact on individual families and many local service 
communities. 

Some limited, anecdotal information about Even Start's impact on the local service 
delivery structure was collected as part of the In-Depth Study. In that component of the 
evaluation, project directors described how some school districts had incorporated aspects 
of Even Start, such as home visits or parent involvement, into the district's philosophy 
and approach. In order to gather more systematic data about community impacts, a 
separate study focusing on that topic would be necessary. 

Even Start is too small a program to make a major impact on its entire target population. 
In spite of this caveat it is important to take note of several areas which indicate that 
Even Start has had a broad impact. First, Even Start funding has grown substantially 
each year since 1989. In a time of fiscal austerity, this is a clear and visible signal that 
Even Start proponents have been able to make a convincing case for the benefits of their 
program. Second, Even Start projects now exist in every state in the nation -- a further 
indication of Even Start's widespread support. Third, anecdotal evidence Fhows that local 
Even Start projects have generated a wealth of valuable and practical ideas about 
implementing family literacy programs -- ideas which have helped other program 
developers. 

How well does the basic Even Start model work? 

Even Start has been well-implemented over a period of years, leading to short-term 
improvements in the lives of many adults and their children. Based on the findings from 
this evaluation, we can draw several conclusions about the viability of the basic Even 
Start model: 

• The model takes multiple years to fully implement. While 
implementation is not easy, almost all projects are able to offer all 
three core services and multiple support services, to offer some 
home-based services, and some services to parents and children 
together, and to make appropriate collaborative arrangements with 
local service providers. 
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• As they matured in their third and fourth years, Even Start projects 
were able to use the same amount of resources to recruit and serve 
about twice as many families as they served in their first year. 

• The total amount (i.e., number of hours of participation} of core 
instructional services received by participating families is important. 
A larger amount of participation in core services is associated with 
larger test gains for adults and children. 

• Participation in adult education services led many Even Start adults 
to attain a GED certificate. 

• Participation in early childhood education led Even Start children to 
attain school readiness skills sooner than they otherwise would 
have. 

• Greater parents' participation in parenting education is related to 
increases in their children's vocabularies. 

• Even Start did not produce any measurable effects on parenting 
skills, family resources, income, or employment. 

• Program retention is increased in projects that provide larger 
amounts of home-based services. 



The current evaluation has identified areas where Even Start has had short-term effects 
on children and parents. However, in order to evaluate long-term effects on program 
participants, a longitudinal study is required. For example, the long-term effect of Eve,. 
Start on children's school behaviors and cognitive performance is a critical element in 
assessing Even Start's impact. To investigate this, a study is needed that follows Even 
Start children into the public schools and collects information from teachers, parents, and 
students. 
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